Validating Simulation Models: A General Framework and Four Applied Examples

This paper provides a framework for discussing the empirical validation of simulation models of market phenomena, in particular of agent-based computational economics models. Such validation is difficult, perhaps because of their complexity; moreover, simulations can prove existence, but not in general necessity. The paper highlights the Energy Modeling Forum’s benchmarking studies as an exemplar for simulators. A market of competing coffee brands is used to discuss the purposes and practices of simulation, including explanation. The paper discusses measures of complexity, and derives the functional complexity of an implementation of Schelling’s segregation model. Finally, the paper discusses how courts might be convinced to trust simulation results, especially in merger policy. Acknowledgments: We thank Rich Burton, Yaneer Bar-Yam, Ian Wilkinson, Shayne Gary, Peter McBurney, Hill Huntington, Leigh Tesfatsion, the participants at the Fourth UCLA Lake Arrowhead Conference on Human Complex Systems, and two anonymous referees for their comments and assistance. JEL codes: C63, C15 Ke ywords: simulation; validation; agent-based computational economics; antitrust; functional complexity; Schelling; sufficiency; Daubert criteria Appearing in Computational Economics, 30(3): 265−290, October, 2007. http://www.springerlink.com/content/l3111153q6278403/?p=470f38ced07044e4b7afb2c5e5f400a9&pi=11 © Springer Science+Business Media, LLC.